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EXECUTIVE SUMMARY 

Despite Naval Aviation’s success in cutting its Class A Flight Mishap (FM) rate 
in half each successive decade between 1950 and 1990, the number of aircraft losses 
attributable to human error has remained relatively constant in the last decade. In 
attempting to meet its stated goal of cutting the Naval Aviation’s Class A FM rate 
attributed to human factors in half by FY2000, the Human Factors Quality Management 
Board’s (HFQMB) initial focus on aircrew error has generated positive results. Since the 
HFQMB’s inception in 1996, the U.S. Navy Class A FM rate dropped to its lowest point 
in FY 1997 and the U.S. Marine Corps posted its lowest rate in FY 1998. Unfortunately, 
the singular focus on reducing aircrew error has failed to attain the overall goal. Over the 
past decade, approximately one in five Naval Aviation Class A FMs were determined to 
have maintenance error as a casual factor. The HFQMB’s prior success, coupled with a 
Naval Air Systems Command drive to address ongoing aircraft maintenance issues, has 
compelled the HFQMB to further analyze aviation maintenance in order to assist in 
reaching its stated goal. 

The HFQMB is currently shifting gears to aviation maintenance and has 
established a Human Factors in Maintenance and Material (HFMM) Process Action 
Team (PAT). The HFMM PAT adapted the HFQMB’s three-pronged approach to further 
analyze maintenance-related errors and to find systems or processes to eliminate them. 
The three-pronged approach included extensive mishap data analysis, benchmarking, and 
a climate assessment survey. The PAT’s initial concentration was on major Class A 
mishaps and common latent conditions that contributed to the active failures that caused 


major Maintenance Related Mishaps (MRM). Schmorrow applied a maintenance 
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extension to the Human Factors Analysis and Classification System (HFACS) while 
examining 470 Naval Aviation MRMs in order to identify the most likely forms of 
maintenance error and the conditions associated with those errors. The HFACS 
maintenance extension categories of error-- squadron, violation, unforeseen, and crew 
resource management -- were found to be the most significant contributors in terms of 
cost. In analyzing 124 Naval Aviation incidents, which occurred during FY90-FY98, 
Teeters discovers two initial areas for immediate consideration: contractor related 
incidents and those incidents with procedural violations. The PAT also benchmarked 
Other organizations and programs, which soon led to the creation and implementation of 
the Groundcrew Coordination Training (GCT). Lastly, the Maintenance Climate 
Assessment Survey (MCAS) was created and implemented in order to gain valuable 
insight into the maintenance community’s perception of its safety climate and culture. 
This thesis is the final piece of the mishap data analysis, benchmarking, and 
climate assessment tripod currently being conducted at the request of the Commander, 
Fleet Logistic Support Wing. The study has nearly 1000 participants from three different 
aircraft communities that comprise the FLSW. The results of this study conclude that the 
MCAS can be utilized as a tool for effectively capturing an aviation maintainer’s 
perceptions of safety in maintenance operations. Through analysis of the MCAS it is 
found that the components of Cummunication/Functional Relationships and Risk 
Management pose the most immediate concern to the aviation maintainers within the 
FLSW. Thus, these areas would serve as good starting points for mitigating risk in 


aviation maintenance operations. Overall, the general safety climate of the FLSW is 
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assessed to be good; however, some potential areas within each community have been 
identified for focusing and prioritizing safety intervention efforts. 

Through Analysis of Variance (ANOVA) and Multiple Comparison testing this 
study shows that the mean responses of the C-9 community are statistically different than 
the mean responses of both the C-130 and C-20 aircraft communities that comprise the 
FLSW based upon a model of High-Reliability Organizations (HROs). Some generalized 
observations can be made in analyzing the Model of Organizational Safety Effectiveness 
(MOSE) component mean responses. Almost across the board the C-9 community’s 
mean responses to questions are noticeably higher than those of both the C-130 and C-20 
communities. Moreover, the culture of the C-130 maintenance community is very 
sensitive to questions that pertain to issues of being over-committed, under- 
manned/under-staffed, burdening disproportionate workloads, the eroding effects of 
personnel turnover, and the negative impact of collateral duties acting upon maintenance 
safety. The old adage of accomplishing more with less seems to have worn thin within 
the C-130 maintenance community. Additionally, the C-20 maintenance community 
seems to express concern with coordination, functional relationships, and communication 
related issues. Lastly, statistical significance is also found along the MOSE components 
which may also assist in helping to prioritize intervention efforts within the FLSW. 

Naval Aviation does not need to wait until another aircraft mishap occurs until it 
attempts to understand what is going on within the safety climate/culture of its squadrons. 
The MCAS is best suited to be utilized as a proactive tool by Squadron Commanders and 
Aviation Safety Officers (ASOs) to identify, prioritize, and focus their safety intervention 
efforts in order to continue to reduce human error in aviation maintenance operations. 
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Lastly, a proposed revised set of MCAS questions is enclosed for future use by 
Commanders and ASOs as a tool to identify the most urgent process problems in ongoing 


aviation maintenance operations. 
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Jf INTRODUCTION 


Despite Naval Aviation’s success in cutting its Class A Flight Mishap (FM) rate 
in half each successive decade between 1950 and 1990 (NPS, School of Aviation Safety. 
1998), the proportion of aircraft losses attributable to human error has remained relatively 
constant in recent years (Nutwell & Sherman, 1997). Following the tragic 1996 
Nashville, TN F-14 mishap, senior Naval Leadership established a Human Factors 
Quality Management Board (HFQMB) with a goal of cutting Naval Aviation’s Class A 
FM rate attributed to human error in half by Fiscal Year (FY) 2000 (HFQMB, 1997). In 
an attempt to meet this goal, the HFQMB focused initially on aircrew error, which was 
found to be a causal factor in three out of every five Class A FMs. Using an approach 
similar to that in investigating aircrew error, the HFQMB have since expanded their focus 
to investigating human factors maintenance-related errors. This thesis contributes to this 
investigation by capturing the maintainers’ perspectives of risk management, safety 


climate and culture within the aviation maintenance community. 


A. OVERVIEW 

The HFQMB’s (1997) strategy consists of concentrated effort in three areas of 
evaluation: (1) Mishap Data Analysis, (2) Organizational Benchmarking, and (3) 
Command Safety Assessment. The objective of analyzing mishap data is to identify the 
human factors issues in past Class A FM cases. The HFQMB identify both aircrew and 
supervisory error as major contributing factors. The subsequent analysis allowed the 
HFQMB to prioritize the intervention target areas based upon prevaling human errors 
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contributing most to FMs. The benchmarking effort uncovers the best practices and 
processes utilized in other aviation organizations such as the commercial airlines use of 
line oriented flight training. In short order the Aircrew Coordination Training (ACT) 
program was being targeted to give aircrews a means to perform Computer Aided 
Debriefs (CADs) following aircrew simulation training (Nutwell & Sherman, 1997). 
Finally, a Command Safety Assessment Survey (CSAS), based on a model of high 
reliability organizations (Libuser 1994; Roberts 1990) was developed to assess a 
command’s safety climate from an aircrew perspective (Ciavarelli & Figlock, 1997). The 
survey’s results indicated that organizational and supervisory issues were seen as 
impacting flight safety (Nutwell & Sherman, 1997). Since the HFQMB’s inception the 
U.S. Navy Class A FM rate dropped to its lowest point in Fiscal Year 1997 (Naval Safety 
Center, 1997) and the U.S. Marine Corps posted its lowest flight mishap rate in Fiscal 
Year 1998 (Naval Safety Center, 1998). These advances have been attributed to the 
HFQMB’s primary focus and efforts to reduce aircrew error, however the proportion of 
mishaps attributable to human error has remained relatively the same (Naval Safety 
Center, 1999). 

In early 1999, the Vice Chief of Naval Operations reaffirmed the drive to 
systematically reduce human error and established a new objective of reducing related 
mishaps from current levels by 50 percent at the end of FY2000 (T. Meyers personal 
communication, January 8, 1999). However, only limited success in reducing human 
error related aviation mishaps can be expected if only aircrew error is being addressed. 


The HFQMB’s success, coupled with a Naval Air Systems Command drive to address 


ongoing aircraft maintenance issues (Lockhardt, 1997), compelled it to start examining 
the maintenance factors that lead to one out of every five Class A FMs. 

The HFQMB, in expanding their focus to include aviation maintenance, formed a 
Human Factors in Maintenance and Material (HFMM) Process Action Team (PAT). The 
HFMM PAT elected to adapt the same three-pronged approach utilized in examining 
aircrew factors; and its initial concentration was on 63 major maintenance-related Class 
A mishaps and the common latent conditions that “set the stage” for active failures that 
cause them (Schmidt, Schmorrow & Hardee, 1998). The results of their study indicate 
that supervisory and climate/cultural issues play a direct role in maintenance-related 
mishaps. Schmorrow (1998) then studies 470 Naval Aviation MRMs ranging from 
mishaps to more routine incidents in order to identify the most common forms of 
maintenance error and the conditions associated with them. Utilizing the same analytical 
process he again found that supervisory and climate/cultural issues play a significant role 
in maintenance error. 

Early maintenance benchmarking initiatives uncovered a maintenance resource 
management program for maintainers similar to the cockpit resource management 
program for aircrew. A Naval Aviation Groundcrew Coordination Training (GCT) 
program was developed and tested producing promising results (Sian, 1997). The GCT 
proved to be a meaningful training program in which participants learned, as measured by 
a content-knowledge test, valuable information that affected real changes in behavior and 
attitudes. Additionally, a three-month follow-up to the GCT training indicated a decrease 


in unsafe behavior exhibited by GCT participants. 


Finally, Baker (1998) develops a Maintenance Climate Assessment Survey 
(MCAS) based upon a model of high reliability organizations in an attempt to capture the 
maintainers’ perspective of risk management, safety climate and culture within the 
aviation maintenance community. After first developing a survey to capture aviation 
maintainers’ attitudes towards maintenance operations, he then successfully reduced the 
prototype 67 question survey to a comprehensive 35 question instrument. This 35 
question variant of the MCAS is applied in this study to identify trends and possible 
intervention opportunities in an effort to further reduce human error in aviation 
maintenance Operations within the Navy’s Fleet Logistics Support Wing. This study also 
parallels concurrent research being conducted by Oneto (1999), that concentrates on 
safety postures across differing aircraft communities within a common Naval Reserve 


Wing. 


B. BACKGROUND 

The Fleet Logistic Support Wing (FLSW) is comprised of fourteen squadrons 
from three very different aircraft communities: the C-9B “Skytrain II,” the C-130T 
“Hercules,” and the C-20 “Gulfstream” that cumulatively fly over 62,000 hours a year all 
over the world. The 27 C-9Bs are the high-speed workhorses of the FLSW with a 
carrying capacity of 90 passengers or 28,000 pounds of cargo at ranges of up to 2,300 
miles. Each of the 20 heavy-lift transport C-130s have a 4,100 mile range and can 
transport 45,000 pounds of cargo or up to 75 passengers to even the most austere 
airstrips. Lastly, for high speed/priority deliveries there are 6 C-20s with carrying 
capacities of 4,500 pounds of cargo or 26 people at ranges of up to 5,200 miles (Peniston, 
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1998). The U.S. Naval Reserve FLSW 1s a vital component of the worldwide logistics 
pipeline that transports Navy and Marine Corps personnel and parts worldwide and 
resupplies Naval military forces in major supply centers and smaller airfields, alike 
(Peniston, 1998). Given decreasing budgets, force reductions, reduced spare part 
inventories and the evolving concept of "precision logistics", a potential Class A FM or 
any other type or severity of incident within the FLSW could have detrimental effects on 
forward deployed forces’ abilities to fulfill their given missions. 

The Commander of the FLSW became aware of the HFMM’s recent efforts and 
requested a systematic assessment of the Wing’s maintenance safety posture. The Naval 
Postgraduate School’s School of Aviation Safety became engaged in a number of efforts 
to assist in this process. Efforts included the Human Factors Analysis and Classification 
System study of 124 MRMs (Teeters, 1999), GCT briefs at each Squadron location and 
follow up workshops, maintenance oriented operational risk management training, and 
the Wing wide administration of the MCAS. Additionally, the results of precious Naval 
Safety Center on-site maintenance safety surveys were also aggregated. The FLSW 
initiative was culminated with a maintenance human factors/risk management workshop 
hosted by the School of Aviation Safety to discuss the results from each effort. The 
findings were used to determine common/prevalent problem areas, prioritizing them 


according to severity/probability, and to develop notional intervention strategies. 


C. PROBLEM STATEMENT 
The Navy’s Fleet Logistics Support Wing has an outstanding mishap record with 
only one Class A mishap in the last 20 years. However, maintenance factors continue to 


5 


be a factor in one out of every five Naval Aviation mishaps. This thesis constitutes the 
final prong of the HFMM’s three-pronged approach currently being conducted at the 
request of the Commander, FLSW. The study utilizes the 35 question variant of the 
MCAS in order to capture and assess the organizational safety climate of the FLSW’s 
aviation maintenance community based upon the model of high reliability organizations 
in order to help identify and reduce human error in Naval Aviation. Accordingly, it 
investigates the following questions: 

1. What is the safety climate of the FLSW based upon a model of 


“high reliability” organizations from the aircraft maintainer’s perspective? 


NO 


Are there discernable differences between the three aircraft communities 
surveyed that comprise the FLSW based upon the “high reliability” 
organization model? 

3. Do the results of the survey match perceptions of actual errors and factors that 
lead up to them? 


4. Can the Maintenance Climate Assessment Survey (MCAS) be improved? 


D. SCOPE AND LIMITATIONS 

The intent of this study is to gain a better insight into the climate and culture of 
the maintenance community of the FLSW in order to better develop intervention 
strategies that would help reduce human error in aviation maintenance. It 1s hoped that 
lessons learned through this study can produce procedures and processes that are both 
directly applicable and beneficial to all of Naval Aviation. For the purpose of this study 
only Naval Aviation maintenance personnel of the squadrons that comprise the FLSW are 
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surveyed. This study includes 13 of the 14 FLSW squadrons; the lone squadron not 
surveyed (VR-51, Kaneoe Bay, HI) utilizes civilian contract maintenance of its aircraft. 

The survey was administered on site and in a group Setting at the various 
participating squadrons. Additionally, the survey was given in conjunction with a 
scheduled maintenance safety presentation on human factors issues in aviation. The 
Squadrons were in various Stages of training and operational tasking at the time of the 
survey being administered. Some Squadrons were in a safety standdown, some were 
taken during a “drill” weekend while still other Squadrons were in “off-drill’” weekend. 
Lastly, some were executing a normal flight schedule routine while other Squadrons 
where in a “no fly” status weekend. The varying operational taskings that the Squadrons 
were Simultaneously undertaking during the administration of the MCAS accounted for 
much of the variance in the number of surveys collected from each Squadron. The 
potential MCAS respondents were briefed on the survey and its purpose and questions 
that arose pertaining to the survey were answered by the survey administer. The surveys 
were then immediately collected upon completion to allow for maximum accountability. 
For the purpose of this study, two squadron’s responses (VR-55 and VR-57) were 
transformed from the 67 item MCAS utilizing the mapping indicated in Baker’s (1998) 
thesis to the current 35 item MCAS. 

Chapter II contains a literature review on investigative scope, accident causation, 
climate, culture and safety, and high reliability organizations. In chapter III the 
methodology used in this study is discussed. Chapter IV presents the results of this 


study. Lastly, chapter V’s conclusions include findings and recommendations. 


E. DEFINITIONS 

This study uses the following definitions (DON, 1989): 

Naval Aircraft. Refers to U.S. Navy, Naval Reserve, U.S. Marine Corps, and U.S. 
Marine Corps Reserve aircraft. 

Mishap. A Naval Aviation mishap is an unplanned event or series of events directly 
involving naval aircraft, which result in $10,000 or greater cumulative damage to naval 
aircraft or personnel injury. 

Mishap Class. Mishap severity classes are based on property damage and personnel 
injury. The following are the definitions of the three major severity classes: 

a. Class A. A mishap in which the total cost of property damage (including all 
aircraft damage) is at least $1,000,000; or a naval aircraft is destroyed or missing; or any 
fatality or permanent total disability of a person occurs with direct involvement of naval 
aircraft. 

b. Class B. A mishap in which the total cost of property damage (including all 
aircraft damage) is between $200,000 and $1,000,000 and/or a permanent partial 
disability occurs and or five or more personnel are hospitalized. 

c. Class C. A mishap in which the total cost of property damage (including all 
aircraft damage) is between $10,000 and $200,000 and/or injury results in one or more 


lost workdays. 


II. LITERATURE REVIEW 
A. OVERVIEW 

In April 1996, Vice Admiral Bennitt established the Human Factors Quality 
Management Board (HFQMB) in order to address the biggest remaining challenge in 
aviation safety- preventing human error. To aid in matching accident causal factors to 
appropriate intervention areas the HFQMB is using a model for human error created by 
the Naval Safety Center (Nutwell & Sherman, 1997). Human Factors Analysis and 
Classification System (HFACS), an adaptation of Reason’s (1990) human error model, is 
applied to classify Naval aircraft mishaps. Early HFACS analysis identified classic 
human error forms described by Reason such as slips, lapses, mistakes, and violations. 
HFACS also identified human errors related to supervisory and aircrew conditions that 
“set the stage” for active human error forms to occur. (Nutwell & Sherman, 1997). The 
application of HFACS to Fiscal Years 1990-1996 Class A Naval Aviation FMs indicated 
that supervisory factors could be identified in over half of the so-called “pilot error” 
mishaps (Ciavarelli & Figlock, 1998). 

Of the three kinds of human activity that are universal in hazardous technologies 
such as Naval Aviation: control under normal conditions, control under emergency 
conditions, and maintenance-related activities, Reason (1997) asserts that the latter of the 
three poses the largest human factors problem. He contends “Errors by pilots, control 
room operators, and other ‘sharp-end’ personnel may add the finishing touches to an 
aren tnonell asst but it is often latent conditions created by maintenance lapses 


that either set the accident sequence in motion or thwart its recovery.” 


The Maintenance Extension (ME) of HFACS as developed by Schmidt, 


Schmorrow, and Hardee (1998) depicts Supervisory, Maintainer and Working Conditions 
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as latent factors that can impact a maintainer’s performance and can contribute to an 
active failure, an Unsafe Maintainer Act. An unsafe maintainer act may immediately or 
eventually lead directly to a mishap or injury. Additionally, the unsafe maintainer act 
may contribute to and become a latent maintenance condition to be consequently dealt 
with by “sharp end” personnel such as aircrew (see Figure 1). The HFACS ME depicts 


the inpact of organizational climate on maintenance safety. 


- 


ORGANIZATIONAL CLIMAT 
o—~ 
Supervisory Maintainer Workplace 
Conditions Conditions Conditions 
S= Maintainer 
Maintenance 
Conditions 
Perey 


Perey 


wisn insuny? 


Figure 1. HFACS Maintenance Extension Diagram. 


B. INVESTIGATIVE SCOPE 
A number of experts within the aviation research community have recently argued 


that there is now an urgent need to complement analyses of individual human error by 
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moving towards an understanding of the role played by broader system factors in 
accidents such as organizational factors. The International Civil Aviation Organization 
(ICAO) states that: 

The late 70’s, 80’s, and 90’s will undoubtedly be remembered as the 

golden era of aviation Human Factors. Cockpit (and then Crew) 

Resource Management (CRM), Line-Oriented Flight Training (LOFT), 

Human Factors training programs, attitude-development programs and 

similar efforts have multiplied, and a sustained campaign to increase the 

awareness of human error in aviation safety has been initiated. But 

much to the consternation of safety practitioners and the entire aviation 

community, human error continues to be at the forefront of accident 

statistics. (p. 1) 

The ICAO authors go on to point out how human error is often precipitated by more 
systematic, background management and organizational factors. 

Adams and Payne (1992) further underscore the point that only limited success in 
reducing pilot-error accident rates will be achieved if the pilot is the only part of the 
operational problem being fixed. Enders (1992) makes the additional observation that 
most aviation accidents have several causes and that to seek a single “probable” cause of 
an event, such as pilot error or maintenance error therefore misses opportunities for 
learning. He further suggests that causes involving “management or supervisory 
inattention at all levels” are the most prevalent category, and perhaps contribute as much 
to accidents as the total numbers of pilot and maintenance errors put together. Johnston 
(1991) took Enders’ one step further in asserting that the need for aircraft accident 
analysts to adopt a wider investigative reality. For example, for those instances in which 


individuals are found to have failed to follow Standard Operating Procedures, any 


conclusion of the type “the accident occurred because X failed to follow procedure Y” 
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should not simply be the end of the investigation, but rather should always be followed 
by the more probing question of “and why was this so?” (Johnston, 1991). 

In Naval Aviation and elsewhere, human error is one of a long-established list of 
“causes” used by the press and accident investigators. Reason (1997) argues that human 
error is a consequence, and not a cause, due to the fact that human errors are shaped and 
provoked by precedent workplace and organizational factors. He explains only by 
understanding the context in which an error was provoked can investigators hope to 
prevent the error’s recurrence explains Reason. There are several stages in the 
development and investigation of an organizational accident (see Figure 2). The 
triangular shape represents the system producing an organizational accident, and it has 
three levels: the person (unsafe acts), the workplace (error-provoking conditions), and the 
organization. The dark upward arrows indicate the direction of causality and the 


downward arrow indicates the investigative steps (Reason, 1997). 







Conditions 
/ Acts 


Local Factors 


Organizational Factors 


Figure 2. Reason’s Stages in Organizational Accident Development/Investigation. 


Clearly, one can begin to sense the nsing ground swell of researchers and analysts 
seeking a new way of conceptualizing and quantifying processes of risk handling and 
management in regards to organizational factors in aviation. HFACS ME is one such 
tool that is currently being explored to better capture and conceptualize those latent 
workplace and organizational factors that contribute to human error. A more in-depth 
understanding of these factors may proactively lead to intervention measures that prevent 


such accidents from ever occurring. 


C. ACCIDENT CAUSATION 

Failures in large-scale technological systems such as aviation cannot be described 
in technical terms alone. Behavioral causes are often predominant in disasters (Pidgeon 
& O’Leary, 1994). Many of the behavioral causes of disasters can be traced to the social 
and organizational arrangements of the socio-technical systems associated with the large- 
scale hazards. Pidgeon and O’ Leary (1994) explain that the concept of a socio-technical 
system stresses the close inter-dependence between a technology and the human 
resources necessary for its use. These social and technical components interact with, and 
over time change each other in complex and often unforeseen ways they further explain. 
It is now clear that the human causes of disasters in complex socio-technical systems 
cannot all be adequately classified under the catch-all term of “human error’ (Pidgeon, 
1991). The behavior contributions to disasters are often more subtle and diverse in nature 
ranging from Reason’s (1990) slips, lapses, mistakes and faulty decision making of 


individuals to more systematic and complex organizational influences. 


The first comprehensive analysis of the social and organizational preconditions to 
disaster in large-scale technological systems is by Turner (1978) in Man-Made Disasters. 
As a result of detailed analysis of 84 major accidents in the United Kingdom over a ten- 
year period he concludes that such events rarely come about for any single reason. 
Turner's (1978) disaster development model focuses in particular upon the information 
difficulties associated with the attempts of individuals and organizations to deal with 
uncertain and ill-structured safety problems. He further concludes that prior to any 
disaster it is typical to find that a number of undesirable events accumulate unnoticed or 
not fully understood, often over a considerable number of years. 

Turner (1978) defines this gradual development of preconditions as the “disaster 
incubation period.” He goes on to state that this period is one in which interacting sets of 
events build up and are resolved either by taking preventative action to remove one or 
more of the undesirable events or are concluded by a “triggering event” as a final critical 
error or a Slightly abnormal operating condition. The “trigger event” results in a 
catastrophe and consequential investigation sheds light on previously unseen background 
factors. However, caution is warranted in that the immediate tngger may be confused 
with the more systemic background causes of a disaster, or may even be taken to be the 
sole cause. Turner states that a further implication of the incubation period 1s that many 
incidents are distinguished from full-scale accidents only by the absence of a suitable 
trigger event, perhaps due to the intervention of chance factors, with the background and 
latent factors remaining constant. Therefore near-miss incidents can and perhaps must be 


interpreted as important warning signals. 


Perrow (1984) provides a second and related account of failures in socio-technical 
systems in his book Normal Accidents. Perrow’s approach is a more technical one that 
draws upon his experiences from a perspective of systems engineering. Perrow identifies 
two general characteristics: complexity (a system can either be complex or linear) and 
coupling (a system’s coupling can be either tight or loose). Systems such as nuclear 
energy and chemical processing, exhibit both high complexity and tight coupling. These 
co-occurrences pose particular control difficulties. Such systems have a small tolerance 
for deviation and, according to Perrow (1984), will be subject almost inevitably to 
“normal” accidents. 

An organization’s position within an operational “safety space” is determined by 
the quality of the processes used to combat its operational hazards explains Reason 
(1997). There will be some positive correlation between an organization’s position 
within the safety space and the number of incidents that befall it. Organizations that he 
finds at the resistant end of the space are likely to suffer fewer incidents within a given 
time frame than those organizations at the vulnerable end. This correlation, however, 
will never be perfect due to chance. Additionally, Reason (1997) contends that when 
incident rates occur at a very low level, as in Naval Aviation, the occurrence or non- 
occurrence of an incident during a given time frame reveals very little about an 
organization’s position within the safety space. Figure 3 shows how hypothetical 
organizations could be distributed within the safety space at any one time. Reason asserts 
their positions within the space are determined by their intrinsic resistance or 


vulnerability to their operating hazards. 


Increasing resistance Increasing vulnerability 





Organizations 


Figure 3. Reason’s (1997) “safety space.” 


D. CLIMATE, CULTURE, AND SAFETY 

Organizational climate, by virtue of being a more salient phenomenon of 
organizational culture, lends itself to direct observation and measurement and thus has 
had a longer research tradition (Schein, 1990). Zohar (1980) discusses various concepts 
of safety climate. She considers safety climate to be a subset of an organization’s overall 
climate. Safety climate refers to the shared perception of an organization’s members that 
the organization’s leaders are genuinely committed to safety of operations, and have 
taken appropriate measures to communicate safety principals, and to ensure adherence to 
safety standards and procedures (Zohar, 1980). Climate is only a surface manifestation 
of culture suggests Schein (1990), and thus research on climate has not enabled 
researchers to delve into the deeper causal aspects of how organizations function. 
Explanations for variations in climate and norms are essential, and it is the need for this 
illusive information that ultimately drives analysts to “deeper” concepts such as culture 


(Schein, 1990). 


Current interest in the term “safety culture” can be traced directly to the accident 
at the Chernobyl nuclear plant in the former Soviet Union and the response of the 
Western nuclear industries to the human and organizational causes of the disaster 
(Pidgeon, 1991). It 1s argued that safety culture represents a new way of conceptualizing 
processes of risk handling and management in organizational and other contexts (Turner, 
Pidgeon, Blockley, & Toft, 1989). Turner et al. (1989) broadly define safety culture as 
the set of beliefs, norms, attitudes, roles and social and technical practices within an 
organization which are concerned with minimizing the exposure of individuals, both 
within and outside of the organization, to conditions considered to be dangerous. A 
safety culture is more than a group of individuals enacting a set of safety guidelines; it is 
a group of individuals guided in their behavior by their joint belief in the importance of 
safety, and their shared understanding that every member willingly upholds the group’s 
safety norms and will support other members to that common end (Merritt & Helmreich, 
1996). 

Merritt and Helmreich (1996) explain that culture gives us cues and clues on how 
to behave in normal and novel situations, thereby making the world less uncertain and 
unpredictable for us. They further describe two important and distinct components of 
culture. The surface culture, or outer layer of culture consists of observable behaviors 
and recognizable physical manifestations such as members’ uniforms, signs and logos, 
and documents. The deep structure, or inner layer of culture, consists of the values, 
beliefs, and assumptions that underlie the surface structure and provide the logic that 


guides the members’ behaviors (Merritt & Helmreich, 1996). 


Schein (1990), in his comprehensive review of organizational culture described 
how culture is the driving force and the guiding principal behind an organization’s goal 
structure, a means to attaining goals, the source of criteria for measuring progress, and 
the origination of methods for correcting deviations from norms and expected outcomes. 
He posits once an organization has evolved into a mature culture because it has had a 
long and rich history, that culture creates the patterns of perception, thought, and feeling 
of every new generation in the organization and, therefore, also “causes” the organization 
to be predisposed to certain kinds of leadership. In that sense the mature group, through 
its culture, also creates it own leaders. An important paradox of mature culture to 
understand, explains Schein (1985) is this: leaders create cultures, but cultures, in turn 
create their next generation Bi leaders. 

Schein (1990) finds culture also molds behavior of individuals through a system 
of rewards, expectations about status, power, authority, established group boundaries for 
inclusion or exclusion, and underlying concepts for managing deviations from norms. He 
adds that culture is learned by individuals who join an organization and is strongly 
influenced by the organization's structure and leadership. The principal factors that 
contribute to an organization's culture include operational criteria for personnel selection, 
formal training practices, explicit and implicit role expectations, and especially the 
actions of leaders as demonstrated by their example. Schein (1990) believes an 
organization's culture is heavily influenced by what leaders pay attention to, and by what 
they express as the core values or expectations of personnel under their supervision. 

Pidgeon and O’ Leary (1994) view the principal cultural unit within which a safety 
culture is assumed to be located is the organization. Within the organization, members 
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will all be selected, trained, and work within a corporate setting that both shapes beliefs 
and regulates behavior. They feel there is a sense in which it is misleading to talk of an 
organization or corporate culture per se. Rather, it is possible to think of the culture of 
small groups of workers or workcenter, of departments, divisions, and organizations 


being both nested within and sometimes overlapping one another. 


E. HIGH RELIABILITY ORGANIZATIONS 

Roberts (1988) and Libuser (1994) explain that High-Reliability Organizations 
(HROs), organizations that operate in a hazardous environment, yet produce very low 
rate of accidents and incidents, which operate effectively and safely have certain key 
characteristics in common. Some of those characteristics are leadership style, sound 
safety management policies, procedure standardization, adequacy of resources and 
staffing, a defined system for risk management, and other factors. While studying on site 
the operations of "safe" technologically complex organizations such as the U.S. Navy’s 
nuclear powered aircraft carriers Roberts (1988) argues that existing organizational 
research is of little help in understanding the organizational processes within such High 
Reliability Organizations (HROs). She (1988) claims that there are significant 
differences in the internal processes of organizations as a function of the degree to which 
their production technologies are perceived as hazardous or the consequences of 
individual failures vary in severity. 

Based upon the earlier research of Roberts, Libuser (1994) developed a Model of 
Organizational Safety Effectiveness (MOSE) that categorized the traits of HROs into five 
components: 1) Process Auditing (PA) - an ongoing system to monitor hazards; 2) 
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Reward System (RS)- expected compensation or disciplinary action used to shape 
behavior; 3) Quality Assurance (QA)- policies and procedures for promoting quality 
performance; 4) Risk Management (RM)- organizational risk perception and mitigation; 
and 5) Command and Control. (CC)-policies and procedures to mitigate risks. Examples 
of conventional organizations requiring and possessing high reliability today are power 
plants, chemical manufacturing plants, oi] refineries, and transportation industries. 

The elements described by Libuser and Roberts map very neatly into Turner (et 
al., 1989)’s original theoretical account of what might constitute a good safety culture. 
Those four principal areas Turner et al. propose are: 1) location of responsibility for 
safety at strategic management level, 2) distributed attitudes of care and concern 
throughout an organization, 3) appropriate norms and rules for handling hazards, and 4) 
on-going reflection upon Safety practice. Additionally, Degani and Weiner (1994) find 
there is a need to have alignment between all levels within a safety culture: philosophy 
(strategic), policy (tactical), procedures (operational), and practices (working). Schein 
(1985, p. 2) who was emphatic about the primacy of leadership influence on the culture 
of an organization would argue that the leaders must burden the responsibility of seeing 
that alignment carried out at all levels: “The only thing of real importance that leaders do 
is to create and manage culture and that the unique talent of leaders is their ability to 


work with culture.” 


F. NAVAL AVIATION AS A HIGH RELIABILITY ORGANIZATION 
Naval Avaition is one of many organizations that is recognized as a HRO. As 
part of the HFQMB’s drive to reduce human error in aviation mishaps, Ciavarelli and 
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Figlock (1997) created a survey instrument, the Command Assessment Survery (CAS). 
Based upon Libuser’s MOSE, the survey attempts to capture and assess the HRO aspect 
of Naval Aviation from an aircrewman's perspective. Based on the analysis of 1,254 
aviators who responded to the survey the CAS’s results indicate that organizational and 
supervisory issues are seen by aircrewmen as impacting flight safety (Ciavarelli & 
Figlock, 19977). 

Baker (1998) modified the CAS and developed a prototype 67-item Maintenance 
Climate Assessment Survey (MCAS) in an attempt to capture and assess safety from the 
perspective of Naval Aviation aircraft maintainers. With the assistance of some experts 
in the field of aviation maintenance 67 questions were initially developed from 155 
candidate questions that were tailored specifically to aviation maintenance. One 
additional maintenance specific: component was created and added to the existing MOSE: 
Communication/Functional Relationships. Through principal component analysis Baker 
finds that his survey data is dominated by one dimension accounting for nearly a third of 
the data’s variance with all six of the MOSE components loading upon the dominant 
component. Through factor analysis and a number of other regression techniques Baker 
is then able to successfully reduce the prototype 67 question survey into a comprehensive 
35 question instrument. It is the 35 question variant of the MCAS developed by Baker 
that is applied in this study to further improve the understanding of the possible 


influences of human factors in aviation maintenance operations within the FLSW. 
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G. SUMMARY 

Latent accident-producing conditions are present now; it 1s not necessary to wait 
for bad events to find out what they are argues Reason (1997). Organizations such as 
Naval Aviation need principal ways of identifying their most urgent process problems in 
order to deploy limited resources in most efficient and timely manner. The MCAS can be 
one such tool. Differences identified in safety climate assessments from commonly held 
perceptions among organizations may prove to be a root source of certain unsafe attitudes 
and behaviors. The early detection of these differences will undoubtedly become an 
integral part of a proactive and vital mishap prevention process. To conclude, the 
question of whether one can effectively measure an organization’s safety climate and take 
corrective action to mend deficient processes prior to the occurrence of a accident, 
continues to be perhaps the most important issue in risk management today, both for 


Naval Aviation and other high-reliability organizations. 
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III. METHODOLOGY 


A. RESEARCH APPROACH 

The intent of this study is to assess the aviation maintainer’s perception on safety 
and to achieve a better understanding of the climate within the organization in which 
he/she operates. This study involves the analysis of data obtained from a Maintenance 
Climate Assessment Survey (MCAS) that is based on an existing model of High- 
Reliability Organizations (HROs). This is done in order to identify factors that may be 
utilized in the improvement of safety in aviation maintenance practices. The data 
analysis in this thesis entails descriptive statistics, principal component analysis, cluster 


analysis, analysis of variance (ANOVA), and multiple comparisons in ANOVA. 


B. DATA COLLECTION 

1. Subjects 

The subjects used in the data collection are Navy officers and enlisted personnel 
involved in aviation maintenance of the Fleet Logistic Support Wing (FLSW); no 
civilians participated in the survey. The survey was administered to 13 of the 14 
Squadrons of the FLSW. The lone squadron not surveyed (VR-51, Kaneoe Bay, HI) 
utilizes civilian contract maintenance of its aircraft; meanwhile, the 13 Squadrons that 
were surveyed are al] maintained by selective reserve and active duty personnel. 
Additionally, those thirteen squadrons represent three different communities of aircraft: 
the C-9B “Skytrain Il,” the C-130T “Hercules,” and the C-20 “Gulfstream.” The survey 
respondents are primarily enlisted Navy personnel assigned to aviation maintenance 
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departments. Listed in Table 1 are the participating Squadrons of the FLWS, their 


locations and aircraft type. 



















Table 1. FLSW Participating Squadrons. 


2. Instrument 

The Maintenance Climate Assessment Survey (MCAS) developed by Baker 
(1998), a self-administered, group survey consisting of fifteen demographic and thirty- 
five maintenance related questions is used. The demographic portion of the survey 
includes questions that gather responses on variables such as respondent’s rank, service, 
work shift, years of service, and years of maintenance experience. It also includes 
inquiries into the sirerion level, rating, age, and current maintenance qualifications of 
the survey participants. Anonymity of respondents are maintained at all times. The 35 
maintenance related questions are generated from the five MOSE components of Process 
Auditing, Reward System, Quality Assurance, Risk Management, Command and Control, 


and the aviation maintenance specific component of Communication/Functional 
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Relationships. The survey utilizes a Likert type five point rating scale with verbal 
anchors as follows: Strongly Disagree, Disagree, Neutral, Agree, and Strongly Agree 
(The 35 item MCAS variant 1s included in Appendix A). 

3. Procedure 

The survey is administered on site and in a group Setting at the various 
participating Squadrons. Additionally, the survey is given in conjunction with a 
scheduled maintenance safety presentation on human factors issues in aviation. The 
Squadrons are in various stages of training and operational tasking at the time of the 
survey being administered. Some Squadrons are in a safety standdown, some are taken 
during a “drill” weekend while still other Squadrons are in “off-drill” weekend. Lastly, 
some are executing a normal flight schedule routine while other Squadrons are in a “no 
fly” status weekend. The varying operational taskings that the Squadrons are 
simultaneously undertaking during the administration of the MCAS account for much of 
the variance in the number of surveys collected from each Squadron. The potential 
MCAS respondents are briefed on the survey and its purpose and questions that arise 
pertaining to the survey are answered by the survey administer. The surveys are then 
immediately collected upon completion to allow for maximum accountability. 
For the purpose of this study, two squadron’s responses (VR-55 and VR-57) are 
transformed from the 67 item MCAS utilizing the mapping indicated in Baker’s (1998) 


thesis to the current 35 item MCAS. 
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C. DATA ANALYSIS 

1. Data Tabulation 

Survey data is manually entered into an Excel spreadsheet. The spreadsheet 
consists of rows of respondents and columns of survey questions (both demographic and 
MOSE component). Demographic questions record mainly bivariate and multivariate 
responses. Component question results are coded into the database by assigning scores of 
1, 2, 3, 4, or 5 corresponding to the Likert scale of “Strongly Disagree,” “Disagree,” 
“Neutral,” “Agree,” and “Strongly Agree,” respectively. If no response was given to the 
question the word “BLANK” is entered. Two questions (questions 5 and 27) in the 35 
item MCAS are negatively anchored. A negatively anchored questions seeks to solicit a 
response of “Strongly Disagree” or “Disagree” if systems addressed in the question are 
functioning in a favorable manner. (A response of “Strongly Agree” to a negatively 
anchored question would raise issues of concern to the subject matter of the question.) 
For clarity of discussion in the remainder of this study both questions’ responses have 
been transposed to reflect responses as if the questions are positively anchored. All data 
is then imported into S-Plus 4.5 for further multivariate statistical data analysis. 

2. Statistical Analysis 

Descriptive non-parametric analysis is conducted on the survey data in order to 
describe basic and general information about the demographic and question results. 
Basic summary Statistics are developed. Some results include distributions of survey 
respondents by age, service, education, rank, and current maintenance qualifications. 
Additionally, frequency of responses, means, and standard deviations are tabulated for 
the MOSE components, individual questions, and aircraft communities. The S-Plus 4.5 
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(Mathsoft, 1997) program is used for multivariate statistical analysis. Principal 
Component and Cluster methodologies such as agglomerative nesting and divisive 
analysis are used to further validate the stability of the MCAS. Analysis of variance 
(ANOVA) and multiple comparisons utilizing Tukey’s procedure for pairwise 
comparison are utilized to compare the effects of the three-leveled factor aircraft 
community and the six-leveled factor MOSE component have on the mean responses of 


the survey’s respondents. 


pag 
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IV. RESULTS 


A. DESCRIPTIVE STATISTICS 

1. Sample 

Nearly 1000 Maintenance Climate Assessment Surveys are collected from 
military personnel serving in the squadrons that comprise the Fleet Logistics Support 
Wing. Of these, 819 surveys indicate some amount of aviation maintenance experience 
on behalf of the respondent. The analysis of these 819 surveys will be addressed in the 
Results section of this thesis. 

2. Demographics 

Descriptive statistics are developed for the survey respondents. A demographic 
breakdown according to respondent’s rank was created for each community’s aviation 
maintenance experienced personnel and is presented in Table 2. Approximately 72% of 
those participating in the MCA Survey are personnel serving in enlisted ranks of E4-E6. 
This seemingly high percentage makes intuitive sense given that the majority of aviation 
maintenance performed and immediate supervision provided are from service members 


of this rank category. 


| Blank | E1-E3_ | E4-E6 | E7-E9_—|_—OOfficer_| 
C20 | 0 10.0%] 4 [7.1%] 42 [75.0%] 9 [16.1%] 1 11.8%] 56 
co | 6 1.2%] 35 | 7.1%] 349 [71.1%] 67 113.6%] 34 | 6.9%| 491 
|GrandTotal] 9 | 1.1%] 62 | 7.6%} 586 [71.6%] 98 [12.0%] 64 |7.8%| 819 | 












Table 2. MCAS Respondents’ Rank By Frequency of Response and Percentage. 
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Junior enlisted, senior enlisted and officer numbers appear to be in proper and 
similar proportion for all communities as well as can be seen in the histogram displayed 
in Figure 4 below. Notice that the communities with the most/least senior enlisted 
participants have the least/most officers. This makes sense in that higher level 
supervision will be provided by either the senior enlisted or officer ranks with an 
aggregate of approximately 20% of the survey respondents coming from either of these 
two rank categories. Histograms of other descriptive demographic variables such as: 
Service Type, Education Level, Rating, Current Maintenance Qualifications, and Age of 


this study’s participants are located in Appendix (B). 
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Figure 4. MCAS Respondents’ Rank by Percentage. 
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B. MOSE COMPONENT STATISTICS 

Table 3 displays the six MOSE categories and respective MCAS questions that 
pertain to each component. Summary statistics from evaluating the MOSE categories are 
displayed in Tables 5 through 12. Each table contains the mean community response for 
each of the questions pertaining to a particular MOSE component and an overall 
community MOSE component mean. Standard deviations indicating dispersion from the 
mean value of each question are computed and are found in Appendix (C). Bar charts of 
responses may prove insightful in helping visualize the proportion of Likert scale 
responses (1 = “Strongly Disagree,” 2 = “Disagree,” 3 = “Neutral/Mixed,” 4 = “Agree,” 
5 = “Strongly Agree’’) per community per MOSE component and are also found in this 


section. 












| = Command & Control 11, 17, 18, 19, 33, 35 
Communication / Functional 5-10. 20; 21h 7-235 
Saat Recta TOmaL 


Table 3. MOSE Categories and Questions. 


1. Process Auditing 
The first MOSE component, Process Auditing (PA), contains four 


questions: 


ail 


1.) My command has a dedicated program that targets individual training 
deficiencies and ensures the uniform enforcement of SOPs among 
maintenance personnel. 


2.) My command monitors maintainer qualifications and support equipment 
licensing. 


6.) My command adequately reviews and updates safety practices, follows 
established standards and maintains quality control, ensuring that all 
maintainers are responsible and accountable for safe maintenance. 

8.) Medical and safety staff are used to help identify, manage, and temporarily 
restrict personnel with personal issues and those who pose a risk to safe 
maintenance in this command. 

All questions are positively answered with a mean range from 3.17 to 4.37 across all 
questions with community means ranging from 3.64 to 3.90 (see Table 4). Additionally, 
the PA component yields the highest combined component mean of all the six MOSE 
categories with a mean value of 3.76. Below in Table 5 are the responses and percentages 
of responses to the PA component questions generated by the aircraft communities that 


comprise the VR Wing. Additionally, Figure 5 is a histogram of the PA component 


responses. 


Component 


io} 2 | 6 | 8 | Mean 
| 3.50 | 4.00 | 3.88 | 317 | 364 | 
3 





364_| 3.96 | 3.86 | 3.25 | 3.68 
[_¢9 | 362 | 437] 410 | 333 | 3.90 


Table 4. Process Auditing Component Means. 
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Likert Scale Responses 
PAST ae aa ee ee ae ee 
|Community] # | % | # | % | # | % | # | % | # [| % |Grand Total 
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_  c20_ | 8 | 36%] 17 17.6% 46 [20.5% 121 154.0%] 32 [14.3%] 224 
[Grand Total{ 60 | 1.9%] 224 | 7.0%| 681 [21.1%] 1591 149.4%] 664 |20.6% 
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Table 5. Process Auditing Component Responses by Frequency and Percentage. 
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Figure 5. Process Auditing Component Responses. 


2. Reward System 
The second MOSE component is Reward System (RS) consisting of questions: 
4.) Unprofessional behavior is not tolerated in the maintenance department. 


13.) Safety concerns or unsafe hazards associated with maintenance/flight line 
operations can be reported without fear of retribution knowing that the W/C, 
Q/A, or M/C supervisors will address and manage them for proper 
corrections. 


14.) Violations of SOP, NAMP guidelines or other procedures are discouraged in 
this command. 


8), 


25.) Violations of SOP, NAMP guidelines, or other procedures are not common 
in my command. 


32.) Personnel are comfortable telling supervisors about personal problems 
including illness. 
All questions are answered favorably by all communities with mean question responses 
ranging from 3.37 to 4.13. The C-20 community’s overall component mean exceeds 3.80 
perhaps indicating that a good reward system is perceived to be in place. The RS 
component also yields the highest combined component mean (tied with PA) with a mean 
value of 3.76 (see Table 6). Figure 6 displays the frequency of responses within the RS 


component. 


Component 


Community| 4 | 13 | 14 |) 25 | 32, | Mean 


C-130, 0 | 3.75 | 3.76 | 3.78 | 352 {| 337 | 3.64 
| C200 | 38.77 | 3.80 | 413 | 371 | 365 | 381 





Table 6. Reward System Component Means. 
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Figure 6. Reward System Component Responses. 
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3. Quality Assurance 
The third MOSE component, Quality Assurance (QA), consists of nine 
questions. Those questions are: 


3.) My command has a reputation for quality maintenance and tool control is 
taken seriously. 


7.) QARs/ CDIs and Maintenance Safety Petty Officer are sought after billets in 
my command. 


12.) Inspectors perform all required actions before sign off. 


16.) My supervisors are aware of individual daily workload requirements and 
recognize safety achievements through rewards and incentives. 


22.) Maintainer staffing is sufficient, 1s equally worked and is equally 
stressed /fatigued from shift to shift. 


26.) Proper tools and equipment are available, serviceable and used and I am 
provided adequate resources (time, personnel) to accomplish my job. 


28.) Required publications are available, current, and used. 

29.) The QA division is respected and CDIs / QARs routinely monitor 
maintenance evolutions ensuring that maintenance gripes are either 
corrected or addressed prior to flight. 

30.) Signing off PQS/JQRs/PARs is taken seriously, not gun decked and 

maintenance quality is as high on detachments as it is in homeport. 
All questions with the exception of question 22 are given favorable responses by all 
communities. Question 22 earns unfavorable marks from the C-130 and C-20 
communities with the C-9 community rating it “neutral”. Mean responses from the 


communities for question 22 range from 2.61 through 3.00. Thus, question 22 identifies 


an area of mutual concern within the VR Wing that should be further explored for 
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intervention efforts. Another question of interest 1s question 3 which yields the highest 
means for all communities within the QA component ranging from 3.93 to 4.37. 
Community means for the overall QA component range from 3.51 to 3.73 (see Table 7.). 


Figure 7 displays the frequency of responses within the QA component. 





Component 
Community| 3 | 7 | 12 | 16 | 2 | & | @ | 2 | 3 | Men 
| C130 | 406 | 32 | 380 | 33 | 261 | 33 | 366 | 38 | 347 | 351 | 
[cm | 3% | 3% | 31 | 341 | 26 | 30 | 3@ | se | 30 | 355 
| co | 457 | 357 | 412 | 353 | 30 | 399 | 377 | 410 | 37 | 37 






Table 7. Quality Assurance Component Means. 
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Figure 7. Quality Assurance Component Responses. 


4. Risk Management 
Table 8 reflects the mean values of responses to the Risk Management (RM) 
MOSE component, which is comprised of five questions. Those questions are: 
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9.) Based upon my command’s current manning and assets, it is not over- 
committed. 


15.) Supervisors are more concerned with proper aircraft maintenance than 
mission completion and do not allow cutting corners to meet operational 
commitments. 

24.) Personnel turnover does not affect my command’s ability to operate safely. 

31.) Safety is an integral part of this command’s maintenance 

planning/flight line operations, where QARs are helpful and the QA 

division is not “feared.” 

34.) Maintainers are never purposely put in an unsafe situation to meet the 

flight schedule. 
Two questions are of particular interest and concern in this component. Question 9 is 
unfavorably responded to by the C-130 community with a mean response of 2.71. 
Additionally, both the C-9 and C-20 communities respond marginally with means of 3.10 
and 3.16 respectively. The relatively low responses to this question across the board are 
of concern. Perhaps of greater concern is the noticeable difference (approximately 0.4) in 
the mean response of C-130 community and the other aircraft communities within VR 
Wing. Question 24 is the other noteworthy question in that both the C-130 and C-9 
communities respond unfavorably with mean responses of 2.88 and 2.93 respectively 
while the C-20 community produces a nearly neutral responses of 3.14. The C-130 
community’s RM component mean is a meager 3.28 while both the C-20 and C-9 
communities’ component means are 3.50. Additionally, the RM component yields the 
second lowest combined component mean of all the six MOSE categories that comprise 


this survey with a mean value of 3.43. Figure 8 displays the frequency of responses 


within the RM component. 


a0 


Component 


po Questions 
|Community | 9 | 15 | 24 | 84 | Mean 


C-130 | 2.71 | 3.56 | 288 | 3.79 | 345 | 3.28 | 
| c-200 | 3.16 | 363 | 314 | 389 | 367 | 350 | 
co {| 3.10 | 380 | 293 | 386 | 383 | 3.50 





Table 8. Risk Management Component Means. 
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Figure 8. Risk Management Component Responses. 


5. Command & Control 
The fifth MOSE component, Command & Control (CC), is made up of six 
questions. Those questions are: 


11.) Maintenance control is effective in managing all maintenance activities, 
coordinating between M/C, W/C, and QA pnior to the incorporations of TDs. 


17.) Supervisors communicate command safety goals programs and 
procedures. 
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18.) W/C supervisors are respected by the maintenance chief/officer. 
19.) Qualified personnel properly supervise all maintenance evolutions and 


maintainers are briefed on the potential hazards associated with 
maintenance activities. 


23.) Multiple job assignments and collateral do not adversely affect 
maintenance. 

35.) Safety education and training in my command are comprehensive and 
effective and the safety department is respected by the supervisors and 
maintainers. 

The mean responses to the questions range from 2.74 to 3.91 as can be seen in Table 9. 
The C-20 community’s mean of 2.96 to question 18 is unfavorable while the respective 
C-130 and C-9 community responses of 3.30 and 3.67 are modestly positive in nature. 
The noticeable and sizeable differences in responses among the three communities may 
suggest the presence of differing cultural attitudes within the maintenance communities. 
Another question of significant interest is question 23. Both the C-130 and C-9 
communities respond unfavorably with mean values of 2.74 and 2.75 respectively while 
the C-20 community records a mean response of 3.14. The relatively low rating given by 
all communities indicates a need to address the perceived adverse effects on maintenance 


by exploring intervention strategies to help mitigate those effects. Figure 9 displays the 


frequency of responses within the CC component. 
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Component 


SS Seictestions || See 
| Community| 17 | 17] 18 | 19 | 23 | 85 | Mean 


C-130 | 3.46 | 368 | 330 | 357 | 274 | 363 | 3.40 
C200 | 3.34 | 3.61 | 2.96 {| 348 | 3814 | 3860 | 3.36 
| C9 | 8.74 | 3.89 | 3.67 | 375 | 275 | 391 | 3.62 | 





Table 9. Command & Control Component Means. 
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Figure 9. Command & Control Component Responses. 


6. Communication/Functional Relationships 
Table 10 displays the last MOSE component, Communication/Functional 
Relationships (CF), which is comprised of six questions: 


5.) My command has no problem with passdown between shifts. (Positively 
anchored) 


10.) Within my unit, good communication flow exists up and down the chain of 
command. 


20.) My supervisor shields me from outside pressures which may affect my work. 
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21.) QARs are never pressured by the maintenance supervisors to sign off a gripe. 


27.) Maintenance Control always troubleshoots aircraft discrepancies. (Positively 
anchored) 


33.) I feel I get all information (internal and external) required to perform my job 
safely, and feel free to report safety violations, unsafe performance or other 
unsafe behavior. 

Question 5, yielded an unfavorable response of 2.63 from the C-20 community while 
both the C-130 and C-9 communities produced mean values of 3.05 and 3.23. Question 
10 is also unfavorably scored by the C-20 community with a response mean of 2.61. 
(Note: One squadron of the C-20 community recorded very unfavorable mean values for 
questions 5 and 10 that not only drove the community means lower but more importantly 
flagged these issues for immediate intervention steps within that particular squadron.) 
Question 20, although not unfavorably responded to, yielded only slightly better than 
neutral ratings across the board. Question 27 is of interest because the C-9 community’s 
response of 3.01 is well below the other communities’ mean responses of 3.45 (C-130) 
and 3.64 (C-20). The CF component yields the lowest combined component mean of all 
the six MOSE categories with a mean value of 3.27 and thus issues contained in this 
component would serve as a good starting point for mitigating risk in aviation 


maintenance operations within the VR Wing. Figure 10 displays the frequency of 


responses within the CF component. 


4] 









J Questions | Component 
| 200| tt 83 | Mean 


Community | 5 | 10 
_¢-130_— | 3.05 | 3.01 | 3.00 | 3.10 | 345 | 369 | 3.220 
| co | 3.23 | 349 | 329 | 340 | 301 | 397 | 340 





Table 10. Communications/Functional Relationships Component Means. 
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Figure 10. Communication/Functional Relationships Component Responses. 


7. MOSE Summary 

Some generalized observations can be made in analyzing the MOSE component 
mean responses. The two MOSE components of greatest concern as identified by 
aviation maintenance personnel of the FLSW while participating in the MCAS are CF 
and RM. The focusing intervention efforts in those two areas should be a priority. 
Almost across the board the C-9 community’s mean responses to questions are noticeably 
higher than those of both the C-130 and C-20 communities. Additionally, the culture of 
the C-130 maintenance community is very sensitive to questions that pertain to issues of 


being over-committed, under-manned/under-staffed, burdening disproportionate 
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workloads, the eroding effects of personnel turnover, and the negative impact of 
collateral duties acting upon maintenance safety. The old adage of accomplishing more 
with less seems to have worn thin within the C-130 maintenance community. Lastly, the 
C-20 maintenance community seems to express concern with coordination, functional 


relationships, and communication related issues. 


C. PRINCIPAL COMPONENT ANALYSIS 

Principal Component Analysis (Hamilton, 1992) is performed on the data in an 
attempt to identify a small set of factors that account for most of the variability of the 
data. Utilizing S-PLUS 4.5, the command “princomp (X, na.action=na.omit, cor = F)” is 
used in this analysis. “X” is a 819 by 35 matrix of survey responses. The code 
“na.action = na.omit” is required in order for S-PLUS to handle cases with missing 
observations. S-PLUS omits all cases with missing values (coded NA) and does not use 
them in the performance of a principal component analysis. Ninety such cases are 
present and thus discarded from the original data due to responses missing in the 35 
maintenance related questions that comprised the second part of the survey. This reduces 
the number of cases to be evaluated in the principal component analysis to 729. The “cor 
= F* implies that the analysis is based on a covariance matrix rather than a correlation 
matrix. The covariance matrix is chosen since the original observations are all on a 
common scale. A scree-plot displays the output of the principal component analysis (see 
Figure 11). This figure displays each component’s contribution to the total variance. 
Above each bar in the plot is the cumulative percentage of variance accounted for as 
successive components are added. Notice that nearly one-third of the data’s variance is 
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accounted for by the first component alone with the next nine components’ diminishing 


contributions accounting for only another third of total variance. This suggests that the 


survey data is dominated by the first component dimension. 


Variances 


10 


0.32 






0.476 0.511 . 0545 0575 0.604 0.63 0.655 
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Figure 11. Principal Component Scree-Plot. 


The principal component loadings are the coefficients of the principal components 


transformation. They provide a convenient summary of the influence of the original 


variables on the principal components (S-Plus 4, 1997). A plot of the variables that load 


on the first five components is generated by utilizing the following S-PLUS code: 


“plot(name$loadings),” where “name” is any name assigned to the principal component 


object. The dollar sign operator ($) attaches the “loadings” property of the principal 


component object for plotting. Figure 12 displays the first five components and the 


variables (survey questions) that load upon each. (Note: a large coefficient (in absolute 


44 


value) corresponds to a high loading, while a coefficient near zero indicates the variable 


has a low loading on the component.) 
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Figure 12. Principal Component Loadings. 


Identifying the six questions that load most heavily upon the most dominant 
component, Component 1, reveals that four of the six MOSE components are represented 
(CF, RM, CC, and RS). By further looking at a printout of the principal component 
loadings it is evident that numerous questions load quite heavily on this orthogonal 
projection with only small perturbations in their values (as perhaps can be inferred by the 
figure above). The next four components all load across at least three MOSE components 
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with Components 3 through 5 loading on the QA, CF, and RM questions in particular. 
Thus, it cannot be said that just any one or two MOSE components contribute to the 
majority of the variance within the survey but rather that all six MOSE components play 


an integral in capturing aviation maintainers’ attitude with respect to safety. 


D. CLUSTERING ANALYSIS 

Clustering analysis is a way of classifying a data set into groups that are cohesive 
but separate (S-Plus 4, 1997). For this study, clustering analysis is performed on the 35 
maintenance-related questions comprising the second portion of survey in order to look 
for trends in how the data is grouping. Two methods by which this can be achieved are 
agglomerative and divisive hierarchical clustering. Though the two methods attempt to 
achieve the same end state, the partitioning of the data into separate groups, their 
respective algorithms operate in polar-opposite fashion. 

1. Agglomerative Nesting (‘“agnes’’) 

A hierarchical agglomerative algorithm constructs a hierarchy of clusterings. 
Considered a “bottom-up” approach, the S-PLUS agglomerative nesting algorithm is 
called “agnes.” At first, each observation (survey question) is a small cluster by itself. 
“Agnes” then yields a sequence of clusterings whereas the two "nearest" clusters are 
combined to form one larger cluster until only one large cluster containing all 
observations remains. The S-PLUS code used for this study is as follows: “‘agnes (daisy 
(t (X)), diss= T)’. The data set is transposed through the code “t (X)” so as to cluster 
across the 35 survey questions, vice the cases. The S-PLUS function “daisy” then 
computes a dissimilarity matrix for the transposed data matrix “X.” The “diss=T”’ 
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parameter implies that a dissimilarity matrix will be used in the algorithm’s computation 
of the distance function. The output of the “agnes” function is displayed in a clustering 
tree diagram with the leaf nodes of the tree being the survey questions. The vertical 
coordinate of where two branches join on the clustering tree equates to the dissimilarity 
between combining clusters. The command “plot (mame)” is used to obtain the clustering 
tree, where “name” is any name assigned to the “agnes” created object. An “agnes” 


clustering tree created on the survey data is shown in Figure 13. 
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Figure 13. ‘“‘agnes” Clustering Tree. 


Additionally, “agnes” outputs an Agglomerative Coefficient (AC). An AC 
measures the amount of clustering structure of the data set. An AC of one (1) would 


indicate a perfect clustering structure and a zero (0) would indicate no clustering 
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structure. The resulting AC produced from performing “agnes” on the data set is 0.38 
indicating a modest clustering structure within the data set. 

2. Divisive Analysis (“diana’’) 

A divisive hierarchical algorithm also constructs a hierarchy of clusterings. 
Considered a “top-down” approach, it begins with one large cluster containing all 
observations and divides until each remaining cluster contains only a single observation. 
Clusters with the largest dissimilarity between any two of its observations are selected to 
split first. To divide a selected cluster, the most disparate item in the group (i.e., the one 
with the highest average dissimilarity to all other observations in the selected cluster) 
initiates a “splinter group” (Kaufman & Rousseeuw, 1990). Then for each item outside 
the splinter group, average distances are computed to all other items and to the splinter 
group. Items that are closer to the splinter group than any other item are added, otherwise 
it is paired with whichever item it was closest to. The process is repeated until each 
cluster contains only a single item. 

The S-PLUS code used to perform the divisive analysis on the data set was 
“diana (daisy (t (X)), diss= T).” The data set is transposed through the code “t (X)” so as 
to cluster across the survey questions, vice the cases. The S-PLUS function “daisy” then 
computes a dissimilarity matrix for the transposed data matrix “X.” The “diss=T”’ 
parameter implies that a dissimilarity matrix will be used in the algorithm’s computation 
of the distance function. The output of the “diana” function is displayed again in a 
clustering tree diagram. The command “plot (mame)” is used to obtain the clustering tree, 
where “name” is any name assigned to the object. A “diana” clustering tree created on 
the survey data is shown in Figure 14. (Note: Likert scale responses to both questions 5 
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and 27 were inverted so that all questions considered during the clustering analysis were 


positively anchored.) The resulting divisive coefficient produced from performing 


“diana” on the data set is 0.46. This is a bit larger than the agglomerative coefficient of 


0.38 produced by “agnes”’. 
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Figure 14. “‘diana’”’ Clustering Tree. 
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A visual comparison of both “agnes” and “diana” clustering trees reveals very 


similar clustering results. Though there is no distinct clustering by MOSE categories, the 


items do group similarly when both agglomerative and divisive hierarchical clustering is 
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performed on the MCAS data set. This similar grouping of the survey questions coupled 
with the modest agglomerative and divisive coefficients does support a clustering 


structure and further advocates the stability of the survey. 


E. ANALYSIS OF VARIANCE 

A two-factor analysis of variance (ANOVA) is run to determine if either the 
aircraft community factor or the MOSE component factor pose any effect on the mean 
survey response. Let the letter “’ denote the number of levels of the first factor of 
interest (factor “A” or community) and “J” denote the number of levels of the second 
factor of interest (factor “B” or MOSE component). With / = 3 (the three aircraft 
communities) and J=6 (the six MOSE components), there are then /*/J = 18 possible cross 
classifications consisting of a level of factor “A” and a level of factor “B”. Each such 
combination is viewed as a treatment and is denoted by Kj. There is only one result for 
each possible Kj and the analysis utilizes a two-factor ANOVA with one observation per 


cell. The factors, levels, and responses are shown in Table 11. 









| RS RM 
3.51 3.40 


Table 11. Community Mean Responses per MOSE Component. 


( S | QA 
| 3.51 





There are two hypotheses of interest in a two-factor ANOVA. The first null 


hypothesis, Hoa , states that the different levels of factor “A” have no effect on the true 
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average response. Likewise, the second null hypothesis, Hog , states that there is no 
effect from factor “B” on the true average response. 

The information in Table 13 is then imported from an Excel spreadsheet into S- 
PLUS 4.5. The S-PLUS code for conducting the ANOVA is “anova ( aov ( Avg ~ factor 
A + factor B, data = X)), where “aov” applies the ANOVA function. The “anova” at the 
beginning of the code provides degrees of freedom, sum of squares, and mean square 
values as well as the F-test statistics and corresponding P-values for the ANOVA output. 
The mean responses listed in Table 13 correspond to the “Avg” parameter which is 
modeled by each aircraft community and MOSE component (“factor A” and “factor B” 
respectively). The “X”’ is the data frame that contains all the information. The 


information output obtained from running the two-factor ANOVA is displayed below: 


Analysis of Variance Table 
Response: Avg 
Terms added sequentially (first to last) 
D£ Sum of Sq Mean Sq F Value Pr(F) 

Comm 2 0.1361446 0.0680723 13.31454 0.001516595 
Comp 5,0, 5325008 90-1065601) (20264725200, 000053879 
Residuals 10 0.0511263 0.0051126 

With an alpha level of 0.05, the P-value for the community factor variable proves 
significant at 0.001517. This results in a rejection of Ho, in favor of the alternate 
hypothesis, which states that the communities show real difference with respect to their 
mean responses to the survey’s MOSE components. The MOSE components also prove 


to be highly significant at 0.000054. This causes us to reject Hog in favor of the claim 


that the different MOSE components do not all yield the same mean responses. 
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F. MULTIPLE ANOVA COMPARISONS 

Since both factors in the two-factor ANOVA reject the null hypotheses it would 
be interesting to find out which of the levels of the several factors are significantly 
different from one another. A multiple comparison procedure such as Tukey’s Procedure 
(or the “T”’ method) will do just that. Utilizing the Studentized range probability 
distribution, confidence intervals for all pairwise comparisons are computed at a selected 
alpha level. In the first case, a pairwise comparison of the three aircraft communities’ 
mean responses will be conducted. The resulting confidence intervals represent the true 
values of all differences UW; - HW; between true treatment means. Each interval that doesn’t 
include the value of zero yields the conclusion that uy; and pW; differ significantly at level a 
level of alpha. 

The S-PLUS code for conducting the multiple comparison procedure is 
“multicomp (A, focus = “factor”, method = “tukey”, plot = T).” “multicomp” is the call 
to the S-PLUS function that performs the multiple comparison test. “A” is the object 
output of the ANOVA run in the previous section on the information from Table #. The 
code “focus = factor” specifies which of the factors to plot (in this case, the aircraft 
communities). Various methods are available in the computation of the comparisons, but 
the “method = “tukey’”’ code explicitly utilizes the Tukey method. The function provides 
the standard default alpha level of .05. The “plot = T” code automatically prints the 


confidence intervals to the graphics device and are displayed in Figure 15. 
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Figure 15. Tukey’s Pairwise Comparison of Aircraft Communities. 


The S-PLUS code “tapply (X [,”response”], X [,”factor’”], mean))” 1s used to 
generate a matrix by applying the “mean” function to the “response” variable (the mean 
responses listed in Table #) over the “factor” levels (the aircraft communities) from the 
data frame ““X.”” Having the aircraft community means of 3.45 (C-130), 3.51 (C-20) and 
3.65 (C-9) arranged in increasing order, the graphical display of the confidence intervals 
(Figure 15) is used to obtain those pairings that contain the value of zero. Those 
components were then underlined as shown in Figure 16. Any pair of aircraft 
communities not underscored by the same line corresponds to a pair of true treatment 
means that are significantly different. Items within the same grouping do not differ 
significantly. Thus, Figure 16 shows that there is statistically proven difference in mean 
responses between the C-9 community and both the C-130 and C-20 communities at an 


alpha level of .05. 
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C-130 C-20 C-9 


Figure 16. Identifying Community Means That Are Statistically Different. 


Now a multiple comparison procedure is run on the other significant factor, 
MOSE components, and its graphical output is displayed below in Figure 17. Once again 
the mean value responses are sorted in ascending order and displayed in Table 12. 
Having the MOSE component mean values arranged in order, the graphical display of the 
confidence intervals (Figure 11) is used to obtain those pairings that contain the value of 
zero. Those components were then underlined as shown in Figure 18. Any pair of MOSE 
components not underscored by the same line corresponds to a pair of true treatment 
means that are significantly different whereas items within the same grouping are found 
not to differ significantly. Figure 18 shows that CF is found to be significantly different 
from all other components other than RM. RM, however, is only significantly different 


from PA and RS. Lastly, CC is statistically different than both CF and RS components. 
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Figure 17. Tukey’s Pairwise MOSE Component Comparisons. 


eo 
| 3.27 | 343 | 346 | 3.60 | 3.74 3.74 
Table 12. MOSE Component Means. 


CF RM CC QA PA_ RS 


Figure 18. Identifying Statistically Different MOSE Components. 








Ns CONCLUSIONS 


A. FINDINGS 

The results of this study conclude that the MCAS can be utilized as a tool for 
effectively capturing an aviation maintainer’s perceptions of safety in maintenance 
operations. Through analysis of the responses a number of different questions are 
identified as raising concern within the different communities. Those questions that 
yielded unfavorable responses or an uncharacteristically low response (question 27 for 
the C-9 community) are recorded in Table 13 along with their respective MOSE 
component. Discussions of the individual questions are included in the MOSE 
Component Analysis section of Chapter [V. Overall, the general safety climate of the 
FLSW is good; however, some potential areas within each community have been 


identified for focusing and prioritizing safety intervention efforts. 
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| ocr | RM | cr | cc | Cr | OA | CO UT ORM 
| Community | 5 | 9 | 10 | 18 | 2 | 2 | 2B | 
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Table 13. Potential Intervention Areas Identified by MCAS. 


Through Analysis of Variance (ANOVA) and Multiple Comparison testing it is 
shown that the mean responses of the C-9 community are statistically different than mean 
responses of both the C-130 and C-20 aircraft communities that comprise the FLS W 
based upon a model of high reliability organizations. Some generalized observations can 


be made in analyzing the MOSE component mean responses. Almost across the board 


ay 


the C-9 community's mean responses to questions are noticeably higher than those of 
both the C-130 and C-20 communities. Moreover, the culture of the C-130 maintenance 
community is very sensitive to questions that pertain to issues of being over-committed, 
under-manned/under-staffed, burdening disproportionate workloads, the eroding effects 
of personnel tumover, and the negative impact of collateral duties acting upon 
maintenance safety. The old adage of accomplishing more with less seems to have worn 
thin within the C-130 maintenance community. Additionally, the C-20 maintenance 
community seems to express concern with coordination, functional relationships, and 
communication related issues. Lastly, statistical significance is found along the MOSE 
components which may also assist in helping to prioritize intervention efforts within the 
FLSW. The components of Cummunication/Functional Relationships and Risk 
Management pose the most immediate concern to the aviation maintainers within the 
FLSW and thus would serve as a good starting point for mitigating risk in aviation 
maintenance operations. 

During the exploratory phase of this study, a few questions were identified for 
restructuring. This was due in part to a poor fusing of questions during the transition 
from the prototype 67 question survey to the 35 question survey utilized in this study. A 
proposed list of revised MCAS questions has been produced with input from the School 
of Aviation Safety, AIRPAC’s Maintenance Training Team, and the Naval Safety Center 


and is included in Appendix E. 
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B. RECOMMENDATIONS 

This study can be used as a starting point for several different follow on studies. 
Ideally, MCAS should now be incorporated within Naval Aviation’s regular active duty 
Wings for timely identification and prioritization of potential safety intervention areas 
within aviation maintenance. Minor modifications of the MCAS could also be made for 
a Marine Corps specific version of MCAS. Further analysis and comparisons could then 
be made among similar aircraft communities that are inherent in both the Navy and 
Marine Corps. Additionally, Marine Corps Reserve aviation units could then be 
contrasted with this study of the Navy’s FLSW. Lastly, a more in-depth study of recent 
Mishap Data Analysis and MCAS results would be interesting to determine to what 


degree there is correlation in perceptions of maintenance safety and actual safety records. 


ao 
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APPENDIX A. 35-ITEM MCAS QUESTIONNAIRE 


Purpose: The purpose of this survey is to try and gain valuable insight into the maintenance community’s 
perception concerning aviation mishaps within the Navy and Marine Corps. Your participation and answers 
will be used as a guide in the Navy’s on-going efforts to lower the aviation mishap rate. 

The first fifteen questions, part I, regard biographical data; information particular to yourself. This 
information will aid in the analysis of your responses. NO attempts will be made to identify individual 
respondents or their organizations. 

Part II has 35 questions pertaining to the maintenance community. Please respond to the questions with 
the answer that most correctly reflects your honest opinion. Using a #2 pencil, completely darken each 
response. 

Thank you in advance for your participation! 


PLEASE RESPOND TO EACH ITEM. 


a Your rank? ( ) BE-1 - E-3 ( )E-4 - E-6 ( )CPO E-7 + ( )Officer 

Zo) YOuT Community? 
VFA Com) nS ( ) VMFA ( ) VE ( ) HS ( ) 
VMA ( ) HC (ie) VP ( ) glans CS) VX ( ) 
VR ( ) VQ ey) VAQ (  ) VAW (*) 

SB. Your designaton-m GO, M52x merc )? TONES 

4. Are you currently a department head? () Yes ( ) No 

5. Your service? ( ) USN ( ) USNR TAR ( ) SELRES ( ) Other 


cae YOURS a hie Or (yee Dx (-) NX (ae eMaead (7) “Other 

7. Total years of service? 

8. Total years of Aviation Maintenance experience? 

9. A-School graduate? ( ) Yes (=) No (oon A 

10. Education level:( ) GED ( ) High School ( )College, #yrs 

11. Unit home location? ( )East coast ( )West Coast ( )Other 

i. YOur rating 2 ( )AD/AM ( ) AE/AT ( )PR/AME ( )AO ( )Other 

13. Your age? (Sy 517 =20 Co)ireZ a =7 5 ( ) 25-30 ( ) 30+ 

14. Current maintenance qualifications? 
( )Safe for Flight ( )QAR 
(Ce ee wnt ( )Supervisor 
( )SPO ( )N /A 


Se DULY : a Shore ( ) Sea 


6] 


Strongly Agree Neutral/ Disagree Strongly 


Agree 


Part J 


1. My command has a dedicated program that ( 
targets individual training deficiencies and ensures 
the uniform enforcement of SOPs among maintenance 


personnel. 

2. My command monitors maintainer qualifications ( 
and support equipment licensing. 

3. My command has a reputation for quality ( 
maintenance and tool control 1s taken seriously. 

4. Unprofessional behavior 1s not tolerated in the ( 

maintenance department. 

5. My command has a problem with passdown ( 
between shifts. 

6. Mycommand adequately reviews and updates ( 


safety practices, follows established standards and 
maintains quality control, ensuring that all maintainers 
are responsible and accountable for safe maintenance. 


7. QARs/ CDIs and Maintenance Safety Petty Officer are ( 
sought after billets in my command. 


8 . Medical and safety staff are used to help identify, ( 
manage, and temporarily restrict personnel with personal 
issues and those who pose a risk to safe maintenance in 
this command. 


9. Based upon my command’s current manning ( 
and assets, it 1s not over-committed. 


10. Within my unit, good communication flow ( 
exists up and down the chain of command. 


11. Maintenance control is effective in managing all ( 
maintenance activities, coordinating between M/C, 
WIC, and QA pnior to the incorporations of TDs. 


12. Inspectors perform all required actions before ( 
sign off. 
13. Safety concerns or unsafe hazards associated ( 


with maintenance/flight line operations can be 

reported without fear of retribution knowing that the 
WIC, Q/A, or M/C supervisors will address and manage 
them for proper corrections. 


A. Violations of SOP, NAMP guidelines or other ( 
procedures are discouraged in this command. 


15. Supervisors are more concerned with proper aircraft ( 


maintenance than mission completion and do not 
allow cutting comers to meet operational commitments. 
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) 


) 


Mixed 


( 


) 


Disagree 


. 


Strongly Agree Neutral/ Disagree Strongly 
Agree Mixed Disagree 


16. My supervisors are aware of individual () ( ) ( ) ( ) (3) 
daily workload requirements and recognize safety 
achievements through rewards and incentives. 


17. Supervisors communicate command safety goals, ( ) ( ) 7 ( ) ( ) 
programs and procedures. 


18. W/C supervisors are respected by the maintenance (a) (7) (5) ( ) oo 
chief/officer. 


19. Qualified personnel properly supervise all ( ( ) ( ) 7 ) 
maintenance evolutions and maintainers are 
briefed on the potential hazards associated with 
maintenance activities. 


20. My supervisor shields me from outside (® (2 ( ) ( ) ( ) 
pressures which may affect my work. 


21. QARs are never pressured by the maintenance (o ( ) i) ( ) ( ) 
supervisors to sign off a gripe. 


22. Maintainer staffing is sufficient, is equally worked and ( ) ( ) ( ) i) (.) 
is equally stressed / fatigued from shift to shift. 


23. Multiple job assignments and collateral duties Cs) () ( ) ( ) () 
do not adversely affect maintenance. 


24. Personnel turnover does not affect my () «> (eas) (G) () 
command’s ability to operate safely. 


25. Violations of SOP, NAMP guidelines, or other (>) Cy Cx) (Ge, (33) 
procedures are not common in my command. 


26. Proper tools and equipment are available, (=) (7 (eu) (3) ( ) 
serviceable and used and 1 am provided adequate 
resources (time, personnel) to accomplish my job. 


27. Maintenance Control never troubleshoots (i) ( ) ( ) Cy () 
aircraft discrepancies. 


28. Required publications are available, current, Cj) ( ) fo) ( ) C) 
and used. 


29. The QA division is respected and CDIs / QARs (3 ) ( ) ) ( ) 
routinely monitor maintenance evolutions ensuring 
that maintenance gripes are either corrected or 
addressed prior to flight. 


30. Signing off PQS/JQRs/PARs is taken seriously, Gs) (f) ee) ( ) ( ) 
not gun decked and maintenance quality is as high 
on detachments as it is in homeport. 


31. Safety is an integral part of this command’s Cy) ( ) ) () ( ) 
maintenance planning/flight line operations, where 
QARs are helpful and the QA division is not “feared”. 
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Strongly Agree Neutral/ Disagree Strongly 


Agree Mixed Disagree 
Personnel are comfortable telling supervisors ( ) (ae: (5 ( ) () 
about personal problems including illness. 
. I feel I get all information (internal and external) (4) ( ) ( ) ( ) ( ) 
required to perform my job safely, and feel free to 
report safety violations, unsafe performance or other 
unsafe behavior. 
Maintainers are never purposely put in an ( ) c ( ) ( ) (ee) 
unsafe situation to meet the flight schedule. 
Safety education and training in my command are ( ) ( ) ( ) ( ) ( ) 


comprehensive and effective and the safety department 
is respected by the supervisors and maintainers. 
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APPENDIX B. DEMOGRAPHIC VARIABLE BAR CHARTS. 
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Figure 19. MCAS Respondent Service Type. 
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Figure 20. MCAS Respondent Education Level. 
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Figure 21. MCAS Respondent Rating. 
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Figure 22. MCAS Respondent Current Maintenance Qualifications. 
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Figure 23. MCAS Respondent Duty Type. 
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Figure 24. MCAS Respondent Age. 
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APPENDIX C. SURVEY QUESTION STANDARD DEVIATIONS 













| Community | 7 | 2) | 6 | 8 
| c-130— | 0.96 | 0.84 | 0.77 | 0.88 | 


Table 14. Process Auditing Standard Deviations. 









po Questions 
| Community | 4 | 13] 14 | 28 





Table 15. Reward System Standard Deviations. 









fea eee | ees MM eeRQUCSONS a | | a uae ee 
| Community | 3 | 7 | 12) | 16 | 22 | Ge 8 
p30 | 0.91 | 096 | 0.85 | 102 | 111 | 1.06 | 1.10 | 083 | 0.98 

p20 | 095 | 100 | 067 | 117 | 123 | 1411 | 094 | 0.79 | 096 | 
po | 079 | 092 | 073 | 099 | 108 | 106 {| 1.09 | 074 | 0.98 | 





Table 16. Quality Assurance Standard Deviations. 












|Community | 9 | 15 | 24 Tt 
po | 108 | 091 | 1.09 | 0.90 | 0.94 


Table 17. Risk Management Standard Deviations. 
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a ee ae ee ee ee 
p €-13000 | 1.01 | 0.85 | 1.10 | 0.87 | 1.03 | 0.90 | 
pc200 | 1.05 | 0.85 | 16 | 0.85 | 1.05 | 0.99 | 
C9 | 090 | 080 | 098 | 085 | 111 | 083 | 


Table 18. Command & Control Standard Deviations. 





| Community | 5 | 10 | 20 | 21 | 27) | 33 
C-130 


Table 19. Communication/Functional Relationship Standard Deviations. 
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APPENDIX D. PROPOSED REVISED MCAS QUESTIONS 


COMPONENT 1: PROCESS AUDITING 


I. 


The command adequately reviews and updates safety practices. 


2. The command monitors maintainer qualifications and has a program that targets 


6. 


training deficiencies. 


3. The command uses safety and medical staff to identify/manage personnel at risk. 
4. 
5. Tool Control is taken seriously in the command and support equipment licensing is 


CDIs/QARs routinely monitor maintenance evolutions. 


closely monitored. 
Signing of PQS/JQRs/PARs is taken seriously and not gundecked. 


COMPONENT 2: REWARD SYSTEM and SAFETY CLIMATE 


— 
° 


Our command climate promotes safe maintenance and flight operations. 
Supervisors discourage SOP, NAMP guideline, or other procedure violations and 
encourage reporting safety concerns without fear of retribution. 

Peer influence discourages SOP, NAMP guideline, or other procedure violations and 
individuals feel free to report safety violations, unsafe performance, or unsafe 
behaviors. 

Violations of SOP, NAMP guidelines, or other procedures are not common in this 
command. 

The command recognizes individual safety achievement through rewards and 
incentives. 

Personnel are comfortable approaching supervisors about personal issues/illness. 
Maintenance Safety Petty Officer, Quality Assurance Representative, and Collateral 
Duty Inspector are sought after billets in the command. 

Unprofessional behavior 1s not tolerated in the command. 


COMPONENT 3: QUALITY ASSURANCE 


Ms 


7g 


a 


The command has a reputation for quality maintenance and has set standards to 
maintain quality control. 

The QA Division and Safety Department are respected in the command and are seen 
as essential to mission accomplishment. 


. QARs/CDIs perform all required actions before sign-off and are never pressured by 


maintenance supervisors. 

Maintenance quality on detachments is the same as that at home station. 
Required publications/tools/equipment are available, current/serviceable, and are 
exclusively used. 

QARs are viewed as helpful, and QA is not “feared” in my command. 


vA 


COMPONENT 4: RISK MANAGEMENT 


No 


Multiple job assignments and collateral duties adversely affect maintenance. 

Safety is part of maintenance planning, and additional training/support is provided as 
needed. 

Supervisors recognize unsafe conditions and manage the hazards associated with 
maintenance and flight line operations. 

I am provided adequate resources (time, personnel and equipment) to accomplish my 
job. 

Personnel turnover does not negatively impacts the command’s ability to operate 
safely. 

Supervisors are more concerned with safely conducting aircraft maintenance than 
meeting flight schedule, and do not permit cutting corners or purposely putting 
maintainers in unsafe Situations. 

Maintainer staffing 1s sufficient from shift to shift, and Day/Night Check have 
proportionately stressful/fatiguing workloads. 

Supervisors shield personnel from outside pressures that may affect their work, and 
are aware of individual workload and personal issues. 

Based upon my command’s current manning/assets, it 1s not over-committed 


COMPONENT 5: COMMAND AND CONTROL 


to 


On 


My command temporarily restricts maintainers who are having personal problems. 
Safety decisions are made at the proper command levels, and CC W/C supervisors 
are respected by the maintenance chief/officer. 

Supervisors communicate command safety goals, programs, and procedures, and are 
actively involved in the safety program and management of safety matters. 
Supervisors set the example for compliance to maintenance standards and ensure 
uniform enforcement of SOPs, NAMP guidelines, and other procedures among 
maintenance personnel. 

In my command, safety is an integral part of all maintenance and flight line 
operations and al! maintainers are responsible and accountable for safe maintenance. 
Safety education and training in my command are comprehensive and effective. 

All maintenance evolutions are properly briefed, supervised, and staffed by qualified 
personnel, including flight line activities such as aircraft moves. 

Maintenance control is effective in managing all maintenance activities. 


COMPONENT 6: COMMUNICATION / FUNCTIONAL RELATIONSHIPS 


E 


Ps 


Good communication flow exists up and down the chain of command and I get all the 
information required to perform my job safely. 

Work center supervisors, division CPOs, QA, and M/C coordinate their actions, 
including the incorporation of TDs. 

My command has effective pass-down between shifts. 
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4. Maintenance Control always troubleshoots aircraft discrepancies and gripes are either 
corrected or addressed prior to flight. 
5. Maintainers are briefed on potential hazards associated with maintenance activities. 


ie 
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